The vision of natural and complex images

Eyes and brains have evolved so as to extract from the natural word the sensible order that is crucial for survival. Accordingly, it makes good sense to consider the ecological basis of vision, arguing more or less implicitly for forces shaping visual organisation and processing that act with evolutionary or developmental timescales. Of course, visual systems are not normally blind to many quite "unnatural" stimuli, for instance to scenes of man-made objects like automobiles; and nor should this really be expected. Visual systems actually perform quite effectively within remarkably broad stimulation constraints, operating at different processing .levels, and technology now allows virtually any complex stimulus to be created to study this performance. The collection of papers in this feature issue is thus centred on the diversity of visual performance, and the underlying mechanisms and representations, as investigated with complex and especially naturalistic stimuli. In approaching the "vision of natural and complex images", perhaps the first question that arises is how to express the visually salient properties of such images, starting with their luminance contrast, in a perceptually and physiologically meaningful manner. While conventional definitions, such as root-mean-square (R.M.S.) contrast, may at times be convenient, they do not adequately characterise the contrast an animal will see or respond to. In this issue, Tolhurst and Tadmor, and Peli, provide new evidence that a suitable expression for the discrimination (Tolhurst) and appearance (Peli) of contrast in complex images requires a multi-channel bandpass or multi-scale description (Peli, Journal of the Optical Society of America A, 7, 2032-2040, 1990). Tolhurst et al. point out that the bandwidths of the channels are not critical, but can range similarly to those typically measured for V1 neurons. Comparing several models, Rohaly et al. reinforce the power of a multi-channel model by predicting observers' detectability of objects embedded in natural images, particularly when applied in combination with contrast gain control. The importance of locally computed contrast for a different kind of complex display is well revealed in Rainville and Kingdom's study. They use compressively sampled sinusoidal gratings, whose detection thresholds are known to be higher than those of their smooth cousins (Burr, Ross & Morrone, Vision Research, 25, 717-727, 1985), and conclude that the elevation is not due to luminance adaptation as previously thought, but rather to the local contrast of individual sample bars (see Pelah, SPIE Proceedings, 2657, 216-223, 1996). The appearance of complex images can provide important clues to the function and processing stages of mechanisms that pool visual information. From the appearance of plaids (spatial sinusoids summed at crossed orientations) in careful adaptation experiments, Georgeson and Meese conclude that visual filters can switch from combining across orientation to combining across spatial frequency, and that orientation filtering and edge detection occur prior to the suppression of binocular rivalry. The processing of spatial frequency in pattern appearance is also examined by Durgin and Huk, in connection with the encoding of texture density as measured by density aftereffects (marked changes in perceived texture density following prolonged viewing). They consider both natural and artificial textures and argue from their results that the dimension of density is distinct from traditional spatial frequency in the way it is perceptually represented. Webster and Mollon investigate the appearance of colour, and estimate an observer's visual adaptive state when he looks about different natural environments. They measure spectral distributions from collections of captured natural scenes, and by application of their model are able to predict qualitatively psychophysical colour matching results. Osorio et al. consider the colour vision of stomatopod crustaceans, asking why these animals have an unusually large number, ten or more, of photoreceptor classes when three seem to be enough for other creatures like honeybees and humans? Following certain modelling assumptions, the authors claim that narrow band receptors can actually outperform more broadly tuned ones in improving colour constancy, albeit at the expense of signal to noise ratio and sensitivity. The spatial structure of receptive fields of neurons in mammalian visual cortex, and how they might have developed in the light of world scenes, is the subject of several papers in this issue. Olshausen and Field follow up on their recent work (Olshausen & Field, Nature, 381, 607-609, 1996) by examining the neurobiological implications of sparse coding, proposing from their model a possible explanation for the weak forms of non-linearity found in the responses of cortical simple cells. In a similar vein, Bell and Sejnowski apply an unsupervised learning algorithm, based on information maximisation principles, to an ensemble of natural images, showing the emergence of filters whose maximally independent outputs resemble the profiles of cortical receptive fields. Shouval et al model how binocular misalignment in a two-eye image environment can lead to the formation in neural networks of orientation